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Abstract. Solving large (and sparse) system of simultaneous linear equations has been (and continues to 
be) a major challenging problem for many real-world engineering/science applications [1-2]. For many 
practical/large-scale problems, the sparse, Symmetrical and Positive Definite (SPD) system of linear 
equations can be conveniently represented in matrix notation as [A] {x} = {b} , where the square 
coefficient matrix [A] and the Right-Hand-Side (RHS) vector {b} are known. The unknown solution 
vector {x} can be efficiently solved by the following step-by-step procedures [1-2]: Reordering phase, 
Matrix Factorization phase, Forward solution phase, and Backward solution phase. 

In this research work, a Game-Based Learning (GBL) approach has been developed to help engineering 
students to understand crucial details about matrix reordering and factorization phases. A “chess-like" 
game has been developed and can be played by either a single player, or two players. Through this 
“chess-like” open-ended game, the players/learners will not only understand the key concepts involved in 
reordering algorithms (based on existing algorithms), but also have the opportunities to “discover new 
algorithms” which are better than existing algorithms. Implementing the proposed “chess-like” game for 
matrix reordering and factorization phases can be enhanced by FLASH [3] computer environments, 
where computer simulation with animated human voice, sound effects, visual/graphical/colorful displays 
of matrix tables, score (or monetary) awards for the best game players, etc. can all be exploited. 
Preliminary demonstrations of the developed GBL approach can be viewed by anyone who has access 
to the internet web-site [4]! 


1. INTRODUCTION 

Solving large (and sparse) system of 
simultaneous linear equations (SLE) has been 
(and continues to be) a major challenging problem 
for many real-world engineering/science 
applications [1-2], In matrix notation, the SLE can 
be represented as: 

[A]{x} = {b} (1) 

where [A] = known coefficient matrix, with 
dimension NxN 

{b} = known right-hand-side (RHS) Nxl vector 
{x} = unknown Nxl vector. 

2. SYMMETRICAL POSITIVE DEFINITE (SPD) 
SLE 

For many practical SLE, the coefficient matrix [A] 
(see Eq.1) is SPD. In this case, efficient 3-step 
Cholesky algorithms [1-2] can be used. 

Step 1 : Matrix Factorization phase 

In this step, the coefficient matrix [A] can be 
decomposed into 

[A} = [U] T {U] (2) 


where [U] is an NxN upper triangular matrix. 

The following simple example will illustrate how to 
find the matrix [U], 

Various terms of the factorized matrix [U] can be 
computed/derived as follows (see Eq. 2): 
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Multiplying 2 matrices on the right-hand-side 
(RHS) of Eq. (3), then equating each upper- 
triangular RHS terms to the corresponding ones 
on the upper-triangular left-hand-side (LHS), one 
gets the following 6 equations for the 6 unknowns 
in the factorized matrix [U] . 

I A A 

^11 = V4l ’^12 = >^13 = ( 4 ) 

U \\ U \\ 

U 22 

W 33 = (-^33 — M 13 — M 23 )* ( 5 ) 


277 



In general, for a general NxN matrix, the diagonal 
and off-diagonal terms of the factorized matrix 
[[/] can be computed from the following formulas: 


U u = 4-ZW 
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Wl + M 22^2 = b 2 -> y 2 = b 2 ~ U \ 2 y\ /u 22 ( 14 ) 

Similarly 

y (15) 

U 33 

In general, one has 

j - 1 

b j -Zv< 


As a quick example, one computes: 

_ ^57 ~ ^15^17 ~^25^27 ~ U 35 U 37 ~ U A5 U il 


Thus, for computing u(i = 5,j = 7), one only 
needs to use the (already computed) data in 
columns # i(=5), and # j(=7) of [U], respectively. 

Step 2: Forward Solution phase 

Substituting Eq. (2) into Eq. (1), one gets: 
[U] T [U]{x} = {b} (9) 


Step 3: Backward Solution phase 

Since [t/] is an upper triangular matrix, Eq. (10) 
can be efficiently solved for the original unknown 


vector {*} , according to the order x N _ 2 > , hence 


Let’s define: 


[U]{x} = {y} 

Then, Eq. (9) becomes: 


the name “backward solution”. 


As a quick example, one has: 


[U] T {y} = {b} (11) 

Since [U] T is a lower triangular matrix, Eq. (11) 
can be efficiently solved for the intermediate 
unknown vector {>>}, according to the order 


■, hence the name “forward solution”. 
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Similarly: x 2 = 


y 2 ~ U 23 X 3 ~ U 2A X A 


As a quick example, one has: 
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In general, one has: 
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Remarks 


(30) 


(a) Amongst the above 3-step Cholesky 

algorithms, factorization phase in step 1 
consumes about 95% of the total SLE solution 
time. 

(b) If the coefficient matrix [A] is symmetrical but 
not necessary positive definite, then the above 
Cholesky algorithms will not be valid. In this case, 

the following LDlI algorithms can be employed: 
[A\ = [L][D][Lf (23) 

For example, 

'A, Ai 4,1 [1 0 oTd„ o oTi i 2 , z 3 ,l (24) 

A 22 A 23 — I 2 , 10 0 D n 0 0 1 Z- 32 

m A 3l A 33 A 33 ~ _L 3l L 32 1__ 0 0 D 33 __ 0 0 1 

Multiplying the 3 matrices on the RHS of Eq. (24), 
then equating the resulting upper-triangular RHS 
terms of Eq. (24) to the corresponding ones on 
the LHS, one obtains the following formulas for 
the “diagonal” [D], and “lower-triangular” [L] 
matrices: 

A=4-E4A, (25) 

*=1 


Thus, the LDlI algorithms can be summarized 
by the following step-by-step procedures 

Stepl: Factorization phase 

[yl] = [Z][Z)][Z] r (23, repeated) 

Step 2: Forward solution and diagonal scaling 
phase 

Substituting Eq. (23) into Eq.(1), one gets: 




mM= M 

Eq. (30) can be efficiently solved for the vector 
{z}, then Eq. (29) can be conveniently (and 

trivially) solved for the vector {y}. 

Step 3: Backward solution phase 

In this step, Eq. (28) can be efficiently solved for 
the original unknown vector {x}. 

3. RE-ORDERING ALGORITHMS FOR 
MINIMIZING FILL-IN TERMS [1,2]. 

During the factorization phase (of Cholesky, or 

LDlI algorithms), many “zero” terms in the 
original/given matrix [A] will become “non-zero” 
terms in the factored matrix [U], These new non- 
zero terms are often called as “fill-in” terms 
(indicated by the symbol F). It is, therefore, highly 
desirable to minimize these fill-in terms, so that 
both computational time/effort and computer 
memory requirements can be substantially 
reduced. For example, the following matrix [A] and 
vector {&}are given: 

112 7 0 0 0 2 

7 110 5 4 3 0 

0 5 88 0 0 1 

0 4 0 66 0 0 

0 3 0 0 44 0 

2 0 1 0 0 11 
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Let’s define: 

[if M=M 

[D)iy} = {z\ 

Then Eq. (27) becomes: 


The Cholesky factorization matrix [U], based on 
( 27 ) the original matrix [A] (see Eq. 31) and Eqs. (6-7), 
can be symbolically computed as: 



( 33 ) 
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In Eq. (33), the symbols “x”, and “F” represents 
the “non-zero” and “Fill-in” terms, respectively. 


The factorized matrix [£/’] can be “symbolically” 
computed from [A'] as: 


In practical applications, however, it is always a 
necessary step to send the original matrix [A] 
through re-ordering algorithms (or subroutines) 
[Refs 1-2] and produce the following integer 
mapping array 


IPERM (new equation #) = {o]d equation #} (34) 


such as, for this example: 



(35) 


Using the above results (see Eq. 35), one will be 
able to construct the following re-arranged 
matrices: 
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(37) 


Now, one would like to solve the following 
modified system of linear equations (SLE) for 




[A']{x} = {f} (38) 

rather than to solve the original SLE (see Eq.1). 
The original unknown vector {x}can be easily 

recovered from {x*}and {IPERM}, shown in 
Eq. (35). 



x 0 0 x 0 x 
0x00x0 
0 0 x 0 x 0 
0 0 0 x x F 
0 0 0 0 x x 
0 0 0 0 0 x 


(39) 


You can clearly see the big benefits of solving the 
SLE shown in Eq. (38), instead of solving the 
original Eq. (1), since the factorized matrix 

[£/*]has only 1 fill-in term (see the symbol “F” in 

Eq. 39), as compared to 6 fill-in-terms occurred in 
the factorized matrix [U] (shown in Eq. 33)! 


4. ON-LINE CHESS-LIKE GAME FOR 
REORDERING/FACTORIZED PHASE [4]. 

Based on the discussions presented in the 
previous section 2 (about factorization phase), 
and section 3 (about reordering phase), one can 
easily see the similar operations between the 
symbolic, numerical factorization and reordering 
phases of sparse SLE. 

In practical computer implementation for the 
solution of SLE, the reordering phase is usually 
conducted first (to produce the mapping between 
“old*-+new” equation numbers, as indicated in the 
integer array IPERM(-) in Eqs. 34-35). 

Then, the sparse “symbolic" factorization phase is 
followed by using either Cholesky Eqs. 6-7, or 

the LDZ r Eqs. 25-26 (without requiring the 
actual/numerical values to be computed). The 
reason is because during the “symbolic 
factorization” phase, one only wishes to find the 
number (and the location) of non-zero “fill-in 
terms”. This “symbolic” factorization process is 
necessary for allocating the “computer memory” 
requirement for the “numerical factorization” 
phase which will actually compute the exact 

numerical values of [£/*], based on the same 
Cholesky Eqs. (6-7) (or the LDlJ Eqs. (25-26)). 

In this work, a chess-like game (shown in Figure 1 
[4]) has been designed with the following 
objectives: 
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Figure 1: A Chess-Like Game For Learning to 
Solve SLE. 


computer environments [3], such as animated 
sound, human voice, motions, graphical colors 
etc... have all been incorporated and programmed 
into the developed game-software to be appealing 
to game players/learners. 

2. In the developed “Chess-Like Game”, fictitious 
monetary (or any kind of 'scoring system”) is 
rewarded (and broadcasted by computer 
animated human voice) to game players based on 
how he/she swaps the node (or equation) 
numbers, and consequently based on how many 
fill-in “F” terms occurred. 


(A) Teaching undergraduate/HS students the 
process how to use the reordering output 

IPERM(-), see Eqs. (34-35) for converting the 
original/given matrix [A], see Eq. (31), into the 

new/modified matrix [ A *] , see Eq. (36). This step 
is reflected in Figure 1, when the "Game Player” 
decides to swap node (or equation) “i” (say i =2) 
with another node (or equation) “j”, and click the 
“CONFIRM” icon! 

Since node “i = 2” is currently connected to nodes 
j = 4, 6, 7, 8; hence swapping node i = 2 with the 
above nodes j will “NOT” change the 
number/pattern of “Fill-in” terms. However, if node 
i = 2 is swapped with node j = 1, or 3, or 5, then 
the fill-in terms pattern may change (for better or 
worse)! 

(B) Helping undergraduate/HS students to 
understand the “symbolic” factorization” phase, by 
symbolically utilizing the Cholesky factorized Eqs. 
(6-7). This step is illustrated in Figure 1, for which 
the “game player” will see (and also hear the 
computer animated sound, and human voice), the 
non-zero terms (including fill-in terms) of the 
original matrix [A] to move to the new locations in 

the new/modified matrix [A'] . 

(C) Helping undergraduate/HS students to 
understand the “numerical factorization” phase, by 
numerically utilizing the same Cholesky factorized 
Eqs. (6-7). 

(D) Teaching undergraduate engineering/science 
students and even high-school (HS) students to 
“understand existing reordering concepts", or 
even to “discover new reordering algorithms” 


5. FURTHER EXPLANATION ON THE 
DEVELOPED GAME 

1. In the above Chess-Like Game, which is 
available on-line [4], powerful features of FLASH 


3. Based on the original/given matrix [A], and 
existing re-ordering algorithms (such as the 
Reverse Cuthill-Mckee, or RCM algorithms [1-2]) 
the number of fill-in (“F”) terms can be computed 
(using RCM algorithms). This internally generated 
information will be used to judge how good the 
players/learners are, and/or broadcast 
“congratulations message” to a particular player 
who discovers new (swapping node) strategies 
which are even better than RCM algorithms! 

4. Initially, the player(s) will select the matrix size 
(8x8, or larger is recommended), and the 
percentage (50%, or larger is suggested) of zero- 
terms (or sparsity of the matrix). Then, “START 
Game” icon will be clicked by the player. 

5. The player will then CLICK one of the selected 
node "i” (or equation) numbers appearing on the 
computer screen. The player will see those nodes 
“j” which are connected to node “i” (based on the 
given/generated matrix [A]). The player then has 
to decide to swap node “i" with one of the possible 
node "j”. After confirming the player’s decision, the 
outcomes/results will be announced by the 
computer animated human voice, and the money- 
award will (or will NOT) be given to the 
players/learners, accordingly. In this software, a 
maximum of $1,000,000 can be earned by the 
player, and the “exact dollar amount” will be 
INVERSELY proportional to the number of fill-in 
terms occurred (as a consequence of the player’s 
decision on how to swap node “i” with another 
node “j”). 

6. The next player will continue to play, with 
his/her move (meaning to swap the ith node with 
the jth node) based on the current best non-zero 
terms pattern of the matrix. 
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Note: 

In order to evaluate the students’ performance on 
our developed “Chess-Like Game” for solving 
SLE, a detailed survey will be conducted for the 
“Numerical Methods” (CEE-305) course (for 3-rd 
year undergraduate engineering students) at the 
end of the Fall’2009 semester. Results of this 
survey should give us information about success, 
the level of player engagement, the average 
scores, etc..., and will be reported in the near 
future. 
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